The present application relates to computing and more specifically to software and accompanying systems and methods for validating success of a software upgrade, patch, or build.
Software upgrade verification and validation methods (also called regression testing methods) are employed in various demanding applications, including during testing, developing, provisioning and using of cloud-based applications and services; testing mobile device software prior to deployment; validating whether a new update to a desktop operating system has succeeded, and so on. Such applications often demand efficient mechanisms for analyzing and verifying upgraded software behavior and data, including detecting software errors or undesirable changes (e.g., improperly displayed user interface display screens); validating software functionality that may result from software updates (e.g., patches) or upgrades; validating data and associated software functionality accessible to a given user after an upgrade, and so on.
In general, sometimes newly upgraded software features or functionality may interfere with prior or preexisting features or functionality. Regression testing may catch newly introduced errors or bugs in upgraded software.
Efficient methods and mechanisms for performing regression testing can be particularly important in enterprise applications, such as Business Intelligence (BI) enterprise applications that leverage associated enterprise cloud services, and where software updates may be particularly frequent and where errors can be particularly costly. Furthermore, costs associated with problematic upgrades can demotivate enterprise software upgrades, leaving many enterprises with potentially compromised legacy software. Problematic software errors or other regressions may occur in response to minor and major enterprise software upgrades or patches, including daily builds.
Conventionally, enterprise software developers, software testers, customers, and support agents may manually test upgraded software and report any newly detected errors or inconsistencies. However, manual testing can be prohibitively error prone, time consuming, and costly.
To enhance regression testing efficiency, regression testing software may be used. However, existing regression testing software typically lacks efficient mechanisms for accurately and consistently uncovering newly introduced issues or problems in upgraded software, especially for cloud-based enterprise software. Furthermore, such regression testing software can be complex and costly to build, maintain, and frequently revise to meet the needs of rapidly changing enterprise computing environments.